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Highlights 

Highlights  of  GAO-14-29,  a  report  to 
congressional  committees 


Why  GAO  Did  This  Study 

To  facilitate  unified  operations  across 
the  services,  DOD  has  provided  JPME 
programs  at  departmental  and  service 
academic  institutions  for  almost  30 
years.  In  July  2012,  the  Director  for 
Joint  Force  Development,  who  reports 
to  the  Chairman  of  the  Joint  Chiefs  of 
Staff,  tasked  the  MECC  to  review 
DOD’s  joint  education  objectives  and 
institutions  to  help  ensure  that 
outcomes  match  requirements  for  the 
strategic  environment  projected  for 
Joint  Force  2020.  Subsequently,  the 
National  Defense  Authorization  Act  for 
Fiscal  Year  2013  mandated  GAO  to 
report  to  Congress  on  the  analytical 
approach  used  by  the  MECC  not  later 
than  90  days  after  the  Director 
submitted  the  MECC’s  report  to  GAO, 
which  the  Director  did  on  July  1 , 201 3. 
In  this  report,  GAO  (1)  identifies  the 
purpose  of  DOD’s  study  of  the  JPME 
program,  and  (2)  assesses  DOD’s 
methodology  used  to  conduct  the  Joint 
Professional  Military  Education  study 
and  its  planning  for  follow-on  actions. 
GAO  analyzed  the  MECC’s  final  report 
and  relevant  planning  documents, 
interviewed  DOD  officials  who 
conducted  portions  of  the  study,  and 
reviewed  leading  practices  for 
evaluating  programs  by  GAO  and  other 
entities. 

What  GAO  Recommends 

GAO  recommends  that  DOD  establish 
well-defined  timeframes  for  conducting 
any  follow-on  actions  and  include 
stakeholders  necessary  for 
implementation,  and  assess  the  costs 
of  implementing  recommendations 
made  in  the  MECC’s  recent  study  of 
joint  professional  military  education. 
DOD  concurred  with  both  of  GAO’s 
recommendations. 


View  GAO-14-29.  For  more  information, 
contact  Brenda  S.  Farrell  at  (202)  512-3604  or 
farrellb@gao.gov. 


JOINT  MILITARY  EDUCATION 

Actions  Needed  to  Implement  DOD 
Recommendations  for  Enhancing  Leadership 
Development 


What  GAO  Found 

The  purpose  of  the  Department  of  Defense’s  (DOD)  study  of  its  Joint 
Professional  Military  Education  (JPME)  program  was  to  identify  (1)  desired 
leader  attributes  as  part  of  the  JPME  career-long  learning  experience  needed  to 
support  DOD's  strategic  vision  and  (2)  any  gaps  in  the  current  educational 
program  to  facilitate  the  development  of  the  leaders  needed  to  achieve  that 
vision.  Specifically,  a  Military  Education  Coordination  Council  (MECC) — following 
direction  from  the  Chairman  of  the  Joint  Chiefs  of  Staff  and  the  Director  for  Joint 
Force  Development — proposed  six  desired  leader  attributes,  including,  for 
example,  the  ability  to  anticipate  and  respond  to  surprise  and  uncertainty,  and 
concluded  that  the  existing  institutional  structure  for  providing  JPME  should  be 
retained.  The  MECC’s  gap  analysis,  however,  indicated  that  in  order  to  support 
the  development  of  these  attributes,  a  greater  emphasis  on  career-long  self- 
directed  learning  is  also  needed,  among  other  things.  In  line  with  its  findings,  the 
MECC  made  21  recommendations  to  improve  the  JPME  program  that  address 
increased  accessibility  of  educational  programs,  changes  to  teaching 
methodologies  and  assessment  mechanisms,  and  enhanced  use  of  technology. 

DOD's  methodology  generally  included  leading  practices  for  assessing  training 
programs,  but  DOD  has  not  yet  fully  planned  for  follow-on  actions  and  engaged 
all  stakeholders,  nor  has  it  assessed  the  costs  of  the  MECC’s  recommendations 
to  provide  decision  makers  with  more  timely  and  complete  information  and  help 
ensure  that  the  study’s  results  are  implemented.  Specifically,  the  MECC 
reviewed  other  related  studies,  conducted  a  gap  analysis  to  identify  gaps  based 
on  existing  and  future  needs,  used  the  best  available  data  and  acknowledged 
limitations — all  practices  identified  by  GAO  and  other  government  agencies  and 
research  institutions  as  leading  practices  for  successful  evaluations  of  training 
and  other  programs.  By  contrast,  DOD  documents  state  that  the  results  of  the 
study  were  intended  to  inform  and  shape  the  fall  2013  academic  year,  but  the 
MECC  did  not  complete  its  study  until  June  24,  2013  and  provided  its  report  to 
GAO  on  July  1,  2013.  Further,  DOD  has  not  yet  identified  milestones  and 
timeframes  for  implementing  all  of  its  recommendations.  Subsequently,  the 
department  developed  an  update  to  inform  actions  for  moving  forward  on  its 
recommendations.  DOD  identified  target  dates  for  completion  not  later  than 
September  2014  for  9  recommendations,  but  did  not  include  interim  milestones, 
and  has  not  yet  developed  target  dates  for  12  recommendations.  In  addition,  the 
MECC  did  not  formalize  plans  to  achieve  the  buy-in  of  all  stakeholders  for 
recommendation  implementation.  Without  this  information,  it  may  be  difficult  for 
DOD  to  ensure  that  stakeholders  agree  on  and  are  accountable  for  implementing 
the  recommendations  in  a  timely  manner.  Finally,  the  MECC  did  not  analyze  the 
costs  or  efficiencies  associated  with  implementing  its  recommendations,  but  it 
identified  5  recommendations  that  could  incur  additional  costs  because  they 
require  further  study.  Leading  practices  such  as  accounting  for  program 
resources  enable  managers  to  better  manage  existing  resources  and  plan  for 
future  programmatic  needs.  Without  cost  data  on  the  study’s  recommendations 
or  plans  to  assess  cost  in  the  near  term  as  part  of  continued  efforts  to  implement 
the  results  of  the  JPME  study,  decision  makers  could  be  hindered  in  determining 
the  most  efficient  allocation  of  departmental  resources  for  JPME. 
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October  23,  2013 
Congressional  Committees 

Professional  military  education  is  key  to  the  development  of  the  nation’s 
armed  forces,  and  it  is  the  quality  of  that  military  education  that 
distinguishes  U.S.  forces  around  the  world.1  As  operations  over  the  last 
decade  have  demonstrated,  military  officers  must  think  critically, 
communicate  well,  conduct  themselves  with  integrity,  and  lead  others  in 
difficult  and  often  dangerous  situations  that  require  forces  from  across  the 
military  departments.  The  passage  of  the  Goldwater-Nichols  Department 
of  Defense  Reorganization  Act  in  1986  (Goldwater-Nichols)  marked  one 
of  the  most  significant  attempts  over  the  past  few  decades  to  coordinate 
service  efforts.2  Among  other  things,  the  act  sought  to  establish  clearer 
lines  of  command  and  control  and  improve  the  ability  of  the  services  to 
work  with  each  other  in  truly  joint,  rather  than  simply  multi-service, 
operations.  In  the  wake  of  Goldwater-Nichols,  a  panel  chaired  by 
Representative  Isaac  N.  “Ike”  Skelton  IV  in  1989  undertook  a 
comprehensive  review  of  joint  professional  military  education  (JPME)  and 
made  a  number  of  specific  recommendations  on  how  the  joint  reforms  of 
Goldwater-Nichols  should  be  institutionalized  among  the  services  through 
officer  in-residence  education.  Since  the  Skelton  Report,  Congress  has 
passed  additional  legislation  affecting  the  professional  military  education 
system,  including  a  requirement  to  create  a  program  of  JPME,  through 
which  military  officers  would  be  educated  on  joint  matters  throughout  their 
careers.3  Over  time,  the  Department  of  Defense  (DOD)  and  others  have 
conducted  studies  that,  to  some  extent,  reviewed  the  cumulative  effects 
of  these  legislative  changes. 

The  commitment  within  DOD  to  JPME  demonstrates  a  major  investment, 
both  of  time  and  money.  Some  of  the  education  programs  are  an 
academic  year  in  length,  which  represents  a  significant  part  of  an  officer’s 


^.S.  Congress,  U.S.  House  of  Representatives,  Committee  on  Armed  Services, 
Subcommittee  on  Oversight  and  Investigations,  Another  Crossroads?  Professional  Military 
Education  Two  Decades  After  the  Goldwater-Nichols  Act  and  the  Skelton  Panel,  H.R.  111- 
4  (Washington,  D.C.:  April  2010). 

2  Pub.  L.  No.  99-433  (1986). 

310  U.S.C.  §2154. 
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career.  To  support  the  academic  programs,  in  fiscal  year  2012,  DOD 
spent  approximately  $47.8  million  on  its  JPME  schools  within  National 
Defense  University  alone.4 

In  July  2012,  DOD’s  Director  for  Joint  Force  Development,  expanding  on 
a  white  paper  signed  by  the  Chairman  of  the  Joint  Chiefs  of  Staff,  directed 
DOD's  Military  Education  Coordination  Council  (MECC)  to  prepare  a 
report  on  the  value  of  the  JPME  program,  the  educational  outcomes 
sought,  and  any  changes  necessary  to  strengthen  the  program’s  value  to 
DOD.  According  to  the  director’s  memo,  this  review  was  necessitated  by 
changes  in  the  security  environment,  the  fiscal  climate,  and  the 
experiences  of  DOD’s  joint  force  since  the  passage  of  Goldwater-Nichols 
in  1986.  Subsequently,  the  National  Defense  Authorization  Act  (NDAA) 
for  Fiscal  Year  2013  required  us  to  report  to  Congress  on  the  analytical 
approach  used  by  the  MECC  for  that  report  not  later  than  90  days  after 
receiving  DOD’s  report  from  the  Director  for  Joint  Force  Development. 5 
Our  report  (1 )  identifies  the  purpose  of  DOD’s  study  of  the  Joint 
Professional  Military  Education  program,  and  (2)  assesses  DOD’s 
methodology  used  to  conduct  the  Joint  Professional  Military  Education 
study  and  its  planning  for  follow-on  actions. 

To  identify  the  purpose  of  DOD’s  study  of  the  JPME  program,  we 
reviewed  the  documents  directing  the  study,  the  minutes  of  the  MECC 
meetings,  interim  briefing  report,  and  the  MECC’s  final  report,  dated  June 
24,  201 3. 6  We  also  interviewed  knowledgeable  DOD  officials  from  the 


According  to  its  Fiscal  Year  2014  Budget  Estimates,  DOD’s  National  Defense  University 
is  the  premier  center  for  Joint  Professional  Military  Education  (JPME)  and  is  under  the 
direction  of  the  Chairman,  Joint  Chiefs  of  Staff.  The  university  conducts  JPME  seminars, 
symposia,  and  professional  development  and  conferencing  for  DOD  and  Congressional 
representatives.  The  National  Defense  University  is  located  at  Fort  McNair  in  Washington, 
D.C.  The  dollar  amount  cited  here,  provided  by  a  DOD  official,  includes  both  direct  and 
reimbursable  expenses  and  includes  expenses  for  the  Joint  Forces  Staff  College  (located 
in  Norfolk,  VA),  the  Dwight  D.  Eisenhower  School  of  National  Security  and  Resource 
Strategy,  the  National  War  College,  and  the  CAPSTONE,  PINNACLE,  and  KEYSTONE 
programs  at  the  National  Defense  University.  It  does  not  include  National  Defense 
University’s  overhead  costs,  nor  does  it  constitute  the  entirety  of  DOD’s  JPME  expenses 
that  are  associated  with  the  military  services’  JPME  schools.  According  to  a  DOD  official, 
it  would  be  difficult  to  determine  what  percentage  of  the  services’  funding  goes  specifically 
to  JPME  because,  in  part,  JPME  is  only  a  portion  of  their  programs. 

5Pub.  L.  No.  112-239,  §  547  (2013). 

6DOD  provided  us  with  a  copy  of  its  final  report  on  July  1,  2013.  We  met  our  mandate  with 
a  draft  copy  of  this  report. 


Page  2 


GAO-14-29  DOD’s  2013  JPME  Study 


Background 


Joint  Professional  Military  Education  Division  within  DOD’s  Joint  Staffs 
Directorate  for  Joint  Force  Development  (J7).  The  Joint  Professional 
Military  Education  Division,  in  conjunction  with  the  MECC,  was 
responsible  for  leading  DOD’s  effort.  To  assess  the  methodology  DOD 
used  to  conduct  the  JPME  study  and  its  planning  for  follow-on  actions,  we 
analyzed  relevant  reports  and  guidance  by  GAO,  the  Office  of 
Management  and  Budget,  the  Office  of  Personnel  Management,  and 
private  sector  organizations  such  as  the  RAND  Corporation,  among 
others.7  We  identified  key  practices  contained  in  these  documents  for 
evaluating  educational  programs  and  information  that  were  applicable  for 
the  purposes  of  this  engagement.  We  then  compared  DOD’s 
methodology  with  these  select  leading  practices  for  educational  program 
evaluation.  We  also  reviewed  documents  identified  by  DOD  officials  as 
project  plans  for  the  study,  as  well  as  documentation  of  the  department’s 
intended  follow-on  actions.  Again,  we  interviewed  DOD  officials  from  the 
Joint  Professional  Military  Education  Division  within  the  Directorate  for 
Joint  Force  Development.  We  also  met  with  officials  across  DOD, 
representing  institutions  such  as  the  National  Defense  University,  the 
U.S.  Army  War  College,  and  the  Joint  Forces  Staff  College,  among 
others,  who  were  identified  as  Team  Leads  for  the  study.  Further  details 
about  our  scope  and  methodology  can  be  found  in  appendix  I. 

We  conducted  this  performance  audit  from  March  through  October  2013 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 


Joint  education  is  provided  to  some  extent  at  all  levels  of  officer  and 
enlisted  professional  military  education.  The  professional  development 
and  career  progression  for  both  officers  and  enlisted  personnel  through 
DOD’s  professional  military  education  is  a  service  responsibility; 
embedded  within  the  professional  military  education  systems  is  a 
program  of  JPME  that  is  overseen  by  the  Joint  Staff. 


7See  appendix  I  for  a  list  of  reports  and  guidance  we  reviewed. 
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For  officers,  this  system  is  designed  to  fulfill  the  educational  requirements 
for  joint  officer  management  as  mandated  by  law.8  The  joint  education 
program  is  intended  to  prepare  aspiring  military  leaders  for  both 
conducting  operations  in  a  coherently  joint  force  and  thinking  their  way 
through  uncertainty.  Officer  JPME  program  courses  are  taught  at  multiple 
sites  across  the  country,  including  the  service  Staff  and  War  Colleges  and 
the  National  Defense  University.9  See  appendix  II  for  a  list  and  map  of 
DOD’s  academic  institutions  where  officer  JPME  courses  are  taught. 

According  to  the  Officer  Professional  Military  Education  Policy 
(OPMEP),10  the  JPME  program  for  officers  comprises  curriculum 
components  in  the  five  levels  of  the  officer  professional  military  education 
system  and  includes  three-statutorily  mandated  levels  of  JPME  designed 
to  progressively  develop  the  knowledge,  analytical  skills,  perspectives, 
and  values  that  are  essential  for  U.S.  officers  to  function  effectively  in 
joint,  interagency,  intergovernmental,  and  multinational  operations.  For 
each  of  those  levels,  the  emphasis  changes  to  provide  instruction  in 
target  areas  to  enhance  leader  attributes  of  attending  service  members  at 
their  specific  rank  and  tenure. 

Within  officer  PME,  the  first  level  of  JPME — precommissioning — provides 
officer  candidate  and  officer  training  school,  Reserve  Officer  Training 
Corps,  and  Military  Service  Academy  students  with  a  basic  foundation  in 
defense  structure,  roles  and  missions  of  other  military  services,  the 
combatant  command  structure,  and  the  nature  of  American  military  power 
and  joint  warfare.  The  second  level  of  JPME — primary — provides  junior 
officers  at  the  0-1  through  0-3  ranks11  with  primary  education  on  the 


810  U.S.C.  §§2151-2157. 

9Active  duty  and  reserve  officers  can  also  satisfy  some  JPME  requirements  by  completing 
the  non-resident  curricula  provided  by  National  Defense  University  and  the  services’ 
schools. 

10Department  of  Defense,  Chairman  of  the  Joint  Chiefs  of  Staff,  Officer  Professional 
Military  Education  Policy,  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  (CJCSI) 

1 800.01  D  (Washington,  D.C.,  Dec.  15,  2011). 

11 0-1  through  0-3  ranks  denote  the  first  through  the  third  levels  of  commissioned  military 
officer  ranking,  the  titles  for  which  vary  among  the  services.  0-1 ,  or  officer  level  1 ,  signifies 
an  Army,  an  Air  Force,  or  a  Marine  Corps  Second  Lieutenant,  or  a  Navy  Ensign.  0-2,  or 
officer  level  2,  signifies  an  Army,  an  Air  Force,  or  a  Marine  Corps  First  Lieutenant,  or  a 
Navy  Lieutenant  Junior  Grade.  0-3,  or  Officer  level  3,  signifies  an  Army,  an  Air  Force,  or  a 
Marine  Corps  Captain,  ora  Navy  Lieutenant. 
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tactical  level  of  war.  This  level  of  JPME  helps  to  foster  an  understanding 
of  the  Joint  Task  Force  combatant  command  structure  and  how  national 
and  joint  systems  support  tactical-level  operations,  among  other  subjects. 

The  third  level  begins  the  first  statutorily-mandated  level  of  JPME 
requirements — JPME  Phase  I.12  This  phase  generally  focuses  on  tactical 
and  operational  levels  of  war,  and  is  typically  attended  by  intermediate- 
level  officers  at  the  0-4  rank,  as  well  as  some  officers  at  the  0-5  and  0-6 
ranks.13  JPME  Phase  I  is  incorporated  into  the  curricula  of  the 
intermediate-  and  senior-level  service  colleges,  as  well  as  other 
appropriate  educational  programs  that  meet  JPME  criteria  and  are 
accredited  by  the  Chairman  of  the  Joint  Chiefs  of  Staff.  This  level  of 
education  provides  officers  the  opportunity  to  gain  better  understanding 
from  a  service  component  perspective  of  joint  force  employment  at  the 
operational  and  tactical  levels  of  war.  The  subject  matter  to  be  covered  by 
JPME  Phase  I  instruction  must,  by  law,  include  at  the  least  (1)  national 
military  strategy;  (2)  joint  planning  at  all  levels  of  war;  (3)  joint  doctrine; 

(4)  joint  command  and  control;  and  (5)  joint  force  and  joint  requirements 
development.14 

The  fourth  level  of  JPME  provides  Phase  II  of  the  statutorily-directed 
JPME  requirements.15  JPME  Phase  II  is  a  follow-on  for  selected 
graduates  of  service  schools  and  other  appropriate  education  programs 
that  complements  and  enhances  the  JPME  Phase  I  instruction.  Phase  II 
is  taught  at  National  Defense  University’s  National  War  College  and 
Dwight  D.  Eisenhower  School  for  National  Security  and  Resource 
Management,  the  Joint  Forces  Staff  College’s  Joint  and  Combined 
Warfighting  School  and  Joint  Advanced  Warfighting  School  to  both 
intermediate-  and  senior-level  (0-5  and  0-6)  students,  and  at  senior-level 


1210  U.S.C.  §  2154(a). 

1 3 

The  0-4  rank  denotes  the  fourth  level  of  commissioned  military  officer  ranking,  the  title 
for  which  varies  among  the  services.  0-4,  or  officer  level  4,  signifies  an  Army,  an  Air 
Force,  or  a  Marine  Corps  Major,  or  a  Navy  Lieutenant  Commander.  The  0-5  and  0-6 
ranks  denote  the  fifth  and  the  sixth  levels  of  commissioned  military  officer  ranking,  the 
titles  for  which  vary  among  the  services.  0-5,  or  officer  level  5,  signifies  an  Army,  an  Air 
Force,  or  a  Marine  Corps  Lieutenant  Colonel,  or  a  Navy  Commander.  0-6,  or  officer  level 
6,  signifies  an  Army,  an  Air  Force,  or  a  Marine  Corps  Colonel,  or  a  Navy  Captain. 

1410  U.S.C.  §§  2151(a),  2154(a)(1). 

1510  U.S.C.  §2155. 


Page  5 


GAO-14-29  DOD's  2013  JPME  Study 


service  colleges  to  senior-level  students,  and  consists  of  courses  on  the 
operational  and  strategic  levels  of  war.  Phase  II  helps  prepare  officers  for 
high-level  policy  and  command  and  staff  responsibilities,  with  a  focus  on 
areas  such  as  national  security  strategy  and  joint  strategic  leadership.  In 
addition  to  the  subjects  specified  in  JPME  Phase  I,  the  curriculum  for 
JPME  Phase  II  must,  by  law,  include:  (1)  national  security  strategy; 

(2)  theater  strategy  and  campaigning;  (3)  joint  planning  processes  and 
systems;  and  (4)  joint,  interagency,  intergovernmental,  and  multinational 
capabilities  and  the  integration  of  those  capabilities.16  This  phase 
completes  the  educational  requirement  for  joint  officer  management. 

The  fifth  and  final  level  of  instruction  is  the  CAPSTONE  course  of  JPME 
for  general/flag  officers17  and  focuses  on  the  operational  and  strategic 
levels  of  war  for  high-level  joint,  interagency,  intergovernmental,  and 
multinational  responsibilities.18  Specifically,  the  CAPSTONE  course 
focuses  on  (1)  the  fundamentals  of  joint  doctrine;  (2)  integrating  elements 
of  national  power  across  military  operations  to  accomplish  security  and 
military  strategies;  and  (3)  how  joint,  interagency,  intergovernmental,  and 
multinational  operations  support  strategic  goals  and  objectives.  This  level 
of  JPME  is  tiered  to  ensure  the  progressive  and  continuous  development 
of  executive  level  officers.  Figure  1  summarizes  the  five  levels  of  JPME. 


1610  U.S.C.  §  2155(c). 

17Gen  era  I/flag  officer  denotes  rank  above  the  0-6  level,  the  titles  for  which  vary  among 
the  services.  An  officer  at  this  rank  in  the  Army,  the  Air  Force,  and  the  Marine  Corps  holds 
the  title  of  Brigadier  General  or  higher.  A  Navy  officer  at  this  level  holds  the  title  of  Rear 
Admiral  Lower  Half  or  higher. 

1810  U.S.C.  §2153. 
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Figure  1 :  The  Five  Levels  of  Officer  Joint  Professional  Military  Education  (JPME) 


©  Precommissioning  I  ©Primary 


©JPME  Phase  I3  O  JPME  Phase  II' 


©CAPSTONE3 


Who  attends: 

•  Officer  Candidate, 

•  Officer  Training  School, 

•  Reserve  Officer  Training 
Corps,  and 

•  Military  Service  Academy 
students 

Course  focus: 

•  Provides  a  basic 
foundation  in  U.S. 
defense  structure,  roles 
and  missions  of  other 
military  sen/ices,  the 
combatant  command 
structure,  and  the  nature 
of  American  military 
power  and  joint  warfare. 


Who  attends: 

•  Junior  Officers  at  the  0-1 
through  0-3  ranks 

Course  focus: 

•  Focus  is  on  the  tactical 
level  of  war. 

•  Helps  foster 
understanding  of  the 
Joint  Task  Force 
combatant  command 
structure  and  how 
national  and  joint 
systems  support 
tactical-level  operations, 
among  other  subjects. 


Who  attends: 

•  Intermediate  Level 
Officers  at  the  0-4  rank 

•  Senior  Level  Officers  at 
the  0-5  and  0-6  ranks 

Course  focus: 

•  Focus  is  on  the  tactical 
and  operational  levels  of 
war. 

•  Required  course 
elements  include: 

•  national  military  strategy 

•  joint  planning  at  all 
levels  of  war 

•  joint  doctrine 

"  joint  command  and 
control 

•  joint  force  and  joint 
requirements 
development 


Who  attends: 

•  Senior  Level  Officers  at 
the  0-5  and  0-6  ranks 

Course  focus: 

•  Focus  is  on  the 
operational  and  strategic 
levels  of  war. 

•  Required  course 
elements  include: 

•  national  security 
strategy 

"  theater  strategy  and 
campaigning 

"  joint  planning  processes 
and  systems 

•  joint,  interagency, 
intergovernmental,  and 
multinational  capabilities 
and  the  integration  of 
those  capabilities  (as 
well  as  subjects 
specified  in  Phase  I) 


Who  attends: 

•  General/flag  officers 

Course  focus: 

•  Focus  is  on  the 
operational  and  strategic 
levels  of  war. 

•  Course  elements  include 
high-level  joint, 
interagency,  intergovern¬ 
mental,  and  multina¬ 
tional  responsibilities. 

•  Other  elements  include: 

•  the  fundamentals  of  joint 
doctrine 

■  integrating  elements  of 
national  power  across 
military  operations  to 
accomplish  security  and 
military  strategies 

•  how  joint,  interagency, 
intergovernmental,  and 
multinational  operations 
support  strategic  goals 
and  objectives 


Source:  GAO  analysis  of  DOD  documents  and  policy. 

aDenotes  statutorily-directed  JPME  course  level. 


In  addition  to  meeting  legislative  requirements  to  provide  education  on 
JPME  matters,  DOD’s  colleges  and  universities  that  provide  academic 
year-long  programs  for  officers  are  Master’s  Degree-granting  institutions. 
According  to  the  Middle  States  Commission  for  Higher  Education,  the 
body  that  accredits  National  Defense  University,  among  others, 
accreditation  is  the  means  of  self-regulation  and  peer  review  adopted  by 
the  educational  community.  The  accrediting  process  is  intended  to 
strengthen  and  sustain  the  quality  and  integrity  of  higher  education, 
making  it  worthy  of  public  confidence  and  minimizing  the  scope  of 
external  control.  Accreditation  by  the  Commission  is  based  on  the  results 
of  institutional  reviews  by  peers  and  colleagues  and  attests  to  the 
judgment  that  the  institution  has  met  the  following  criteria: 

•  that  it  has  a  mission  appropriate  to  higher  education; 

•  that  it  is  guided  by  well-defined  and  appropriate  goals,  including  goals 
for  student  learning; 
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•  that  it  has  established  conditions  and  procedures  under  which  its 
mission  and  goals  can  be  realized; 

•  that  it  assesses  both  institutional  effectiveness  and  student  learning 
outcomes,  and  uses  the  results  for  improvement; 

•  that  it  is  substantially  accomplishing  its  mission  and  goals; 

•  that  it  is  organized,  staffed,  and  supported  so  that  it  can  be  expected 
to  continue  to  accomplish  its  mission  and  goals;  and 

•  that  it  meets  the  Requirements  of  Affiliation  and  accreditation 
standards  of  the  Middle  States  Commission  on  Higher  Education. 

DOD  also  conducts  periodic  assessments  of  the  three  statutorily- 
mandated  levels  of  officer  JPME  to  ensure  that  the  curricula  meet  the 
prescribed  joint  educational  requirements  at  each  level,  and  uses  the 
results  of  these  assessments  to  update  educational  policy  as  appropriate. 

For  enlisted  personnel,  the  Enlisted  Professional  Military  Education 
Policy19  circulates  the  policies,  procedures,  objectives,  and 
responsibilities  for  enlisted  professional  military  education  and  enlisted 
JPME.  According  to  that  policy,  enlisted  professional  development  and 
progression  through  the  enlisted  military  education  continuum  is  a  service 
responsibility.  The  initial  focus  of  enlisted  professional  military  education 
is  military  occupational  specialty  training  and  education  required  to 
produce  enlisted  personnel  capable  of  performing  assigned  tasks. 

Beyond  occupational  specialty  training,  the  Enlisted  Professional  Military 
Education  Policy  states  that  all  enlisted  personnel  also  should  be 
exposed  to  enlisted  JPME  as  they  progress  through  their  respective 
services’  enlisted  professional  military  education  system.  In  addition,  the 
policy  states  that,  for  some  enlisted  personnel,  more  comprehensive  joint 
education  is  required  to  prepare  those  servicemembers  for  specific  joint 
assignments. 

Enlisted  JPME  includes  programs  that  span  an  enlisted  member’s  career 
and  apply  to  all  enlisted  personnel.  Basic  Enlisted  JPME  addresses 
educational  guidelines  that  should  be  completed  by  pay  grade  E-6,20 
while  Career  Enlisted  JPME  addresses  educational  guidelines  for  senior 
enlisted  personnel  in  grades  E-6  or  E-7  and  above.  Beyond  these 


1 Q 

Department  of  Defense,  Chairman  of  the  Joint  Chiefs  of  Staff,  Enlisted  Professional 
Military  Education  Policy,  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  (CJCSI) 
1805.01A  (Oct.  1,2010). 

on 

The  general  pay  grade  structure  for  enlisted  personnel  runs  from  E-1  through  E-9. 
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programs,  Senior  Enlisted  JPME  is  a  stand-alone  Web-based  course 
designed  specifically  for  senior  enlisted  personnel  in  pay  grades  E-6 
through  E-9  who  are  serving  in  or  are  slated  to  serve  in  joint 
organizations.  DOD’s  KEYSTONE  course  exists  to  prepare  command 
senior  enlisted  leaders  for  service  in  flag  level  joint  headquarters  or  joint 
task  force  organizations.  This  course  is  designed  for  personnel  who  are 
serving  in  pay  grade  E-9.  The  focus  of  this  course  is  to  enable  command 
senior  enlisted  leaders  to  think  intuitively  from  a  joint  perspective  while 
serving  in  their  capacity  in  a  general/flag  officer  joint  organization. 

DOD  also  developed  a  program  that,  since  2003,  has  provided  JPME 
specifically  to  reserve  officers.  In  response  to  legislative  direction,21  the 
Joint  Forces  Staff  College  established  a  40-week,  blended  learning 
Advanced  Joint  Professional  Military  Education  program  (AJPME)  that 
consists  of  two  distance  learning  periods  and  two  face-to-face  periods. 
Similar  to  the  active  component  courses,  the  Officer  Professional  Military 
Education  Policy  outlines  the  requirements  for  AJPME.  The  program’s 
mission  is  to  educate  reserve  component  officers  to  plan  and  execute 
joint,  interagency,  intergovernmental,  and  multinational  operations  to 
instill  a  primary  commitment  to  teamwork,  attitudes,  and  perspectives. 
AJPME  builds  on  the  foundation  established  by  the  institutions  teaching 
JPME  Phase  I  and  prepares  reserve  component  officers  (0-4  to  0-6)  for 
joint  duty  assignments. 

DOD’s  JPME  programs  are  approved  by  the  Chairman  of  the  Joint  Chiefs 
of  Staff.  The  Director  for  Joint  Force  Development  retains  responsibility 
to  support  the  Chairman  of  the  Joint  Chiefs  of  Staff  and  the  joint 
warfighter  through  joint  force  development,  in  order  to  advance  the 
operational  effectiveness  of  the  current  and  future  joint  force.  The  Joint 
Staff’s  Directorate  for  Joint  Force  Development  (J7)  oversees  general 
policy  for  the  JPME  programs.  The  MECC,  which  was  tasked  with 
conducting  the  most  recent  DOD-initiated  study  on  JPME,  serves  as  an 
advisory  body  to  the  Director  of  the  Joint  Staff  on  joint  education  issues. 
The  purpose  of  the  MECC  is  to  address  key  educational  issues  of  interest 
to  the  joint  education  community,  promote  cooperation  and  collaboration 
among  the  MECC  member  institutions,  and  coordinate  joint  education 
initiatives.  Its  membership  consists  of  the  MECC  principals  and  a 
supporting  MECC  Working  Group.  These  principals  include  the  Director 


21See  10  U.S.C  §  666. 
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for  Joint  Force  Development;  the  Deputy  Director  of  the  Joint  Staff  for 
Military  Education;  the  presidents,  commandants,  and  directors  of  the 
joint  professional  military  education  colleges  and  service  universities;  and 
the  heads  of  any  other  JPME-accredited  institutions.  Additional 
representatives  from  other  commands  and  organizations  may  be  invited 
to  participate  as  is  appropriate.  The  MECC  Working  Group  is  comprised 
of  dean’s  level/  0-6  representatives  of  the  MECC  principals.  The  Deputy 
Director  of  the  Joint  Education  and  Doctrine  Division  chairs  the  MECC 
Working  Group.  Additionally,  the  Secretary  of  Defense,  service  chiefs, 
and  combatant  commanders  are  invited  to  send  participants  to  the  MECC 
meetings  to  provide  feedback  to  improve  the  educational  process. 


DOD’s  Study  Was 
Intended  to  Identify 
Leader  Attributes, 
Career-Long 
Education  Goals,  and 
Educational  Gaps 


The  purpose  of  DOD’s  issued  study  of  the  JPME  program  was  to  identify 
desired  leader  attributes  by  defining  what  is  needed  from  the  JPME 
career-long  learning  experience  to  support  DOD’s  strategic  vision,  as  well 
as  any  gaps  in  the  current  educational  program  to  facilitate  the 
development  of  the  leaders  needed  to  achieve  that  vision.  The  original 
purpose  and  objectives  for  the  study  were  laid  out  in  a  July  1 6,  201 2, 
white  paper  from  the  Chairman  of  the  Joint  Chiefs  of  Staff  and  a  Joint 
Staff  memorandum  issued  with  the  white  paper. 

In  the  white  paper,  the  Chairman  stated  that  the  study’s  purpose  was  to 
support  the  development  of  the  department’s  leaders  by  fostering  the 
values,  strategic  vision,  and  critical  thinking  skills  needed  to  lead  and 
support  the  development  of  Joint  Force  2020. 22  This  white  paper  also 
identified  four  attributes  that  the  department’s  education  programs  should 
develop  in  its  leaders.  Those  attributes  include  the  following: 


•  the  ability  to  understand  the  security  environment  and  the 
contributions  of  all  elements  of  national  power; 

•  the  ability  to  deal  with  surprise  and  uncertainty; 

•  the  ability  to  anticipate  and  recognize  change  and  lead  transitions; 
and 

•  the  ability  to  operate  on  intent  through  trust,  empowerment,  and 
understanding. 


22Joint  Force  2020 — outlined  in  the  September  10,  2012  Capstone  Concept  for  Joint 
Operations:  Joint  Force  2020 — is  the  concept  behind  the  Chairman’s  plan  for  the  future 
force. 
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The  Chairman’s  white  paper  also  stated  that  other  attributes  for  leader 
development  would  evolve  and  would  need  to  be  aligned  with  future 
operations  and  incorporated  into  curricula  to  help  ensure  that  gaps  are 
identified  and  eliminated. 

The  memorandum,  signed  by  the  Director  for  Joint  Force  Development, 
identified  three  questions  meant  to  shape  the  review.  The  first  question 
focused  on  the  value  of  joint  education,  why  it  is  needed,  and  how  the 
military  will  train  and  educate  its  leaders  to  meet  the  requirements  of  Joint 
Force  2020.  The  second  question  focused  on  the  educational  outcomes 
sought  by  the  department,  specifically  with  regard  to  training  leaders  for 
carrying  out  DOD’s  strategic  vision,  in  order  to  help  ensure  alignment  of 
desired  leader  attributes  with  the  curriculum.  The  third  question  built  upon 
the  first  two  by  focusing  on  any  changes  needed  to  strengthen  and 
achieve  the  value  and  outcomes  DOD  seeks  in  building  leaders  for  Joint 
Force  2020.  The  memorandum  also  instructed  DOD’s  MECC  to  conduct 
the  study. 

Subsequent  to  the  issuance  of  the  white  paper  and  memorandum,  the 
MECC  met  on  August  23,  2012,  to  refine  the  study’s  specific  objectives. 
According  to  officials,  at  the  direction  of  the  Director  for  Joint  Force 
Development,  the  MECC  removed  the  requirement  to  address  the  value 
of  joint  education  and  instead  focused  its  efforts  on  identifying  the 
intended  outcomes  of  the  educational  program  and  actions  needed  to 
achieve  the  outcomes.  The  MECC’s  discussions  also  established  two 
broad  objectives  for  the  study — documented  in  meeting  minutes  and  an 
enclosure — based  on  the  direction  in  the  memorandum  and  further 
direction  received  by  the  Chairman.  The  two  broad  objectives  the  MECC 
established  were  to: 

1 .  define  the  Joint  Education  Continuum23  needed  to  meet  the  joint 
leader  development  goals  for  Joint  Force  2020,  and 

2.  determine  the  gaps  that  need  to  be  addressed  for  the  Joint  Education 
enterprise  to  progress  from  the  current  joint  leader  development 
outcomes  to  producing  the  future  desired  joint  leader  attributes  for 
Joint  Force  2020. 


oo 

The  Officer  Professional  Military  Education  continuum,  and  similarly  the  Enlisted 
Professional  Military  Education  continuum,  is  a  graphic  representation  of  key  elements  of 
career-long  officer  (and  enlisted)  education  that  takes  into  consideration  educational 
institutions  and  course  curricula. 
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The  evolution  of  the  study’s  objectives  is  described  below  in  figure  2. 


Figure  2:  Evolution  of  Joint  Professional  Military  Education  (JPME)  Study  Objectives 


The  Director  tor  Joint  Force  Develop¬ 
ment's  July  16,  2012  memorandum 
called  for  a  3-pronged  approach  to 
JPME  study  focusing  on: 

1  The  value  of  joint  education. 

2.  The  educational  outcomes  DOD 
seeks,  and 

3.  The  changes  necessary  to 
strengthen  the  value  proposition 
and  achieve  the  sought  after 
outcomes 


The  Military  Education  Coordination 
Council  (MECC)  met  in  August  2012  to 
discuss  study  objectives  The  Director 
instructed  the  group  to  eliminate  the 
value  element  of  the  study 


MECC  discussions  resulted  in  the  development  of  two 
objectives  for  the  study,  based  on  the  remaining  two 
elements  identified  in  the  memorandum  The  two 
broad  objectives  the  MECC  established  were  to: 

1 .  Define  the  Joint  Education  Continuum  needed  to 
meet  the  joint  leader  development  goals  for  Joint 
Force  2020,  and 

2  Determine  the  gaps  that  need  to  be  addressed  for 
the  Joint  Education  enterpnse  to  progress  from  the 
current  joint  leader  development  outcomes  to 
producing  the  future  desired  joint  leader  attnbutes 
for  Joint  Force  2020 


Source  GAO  analysis  of  DOD  information 


The  MECC’s  first  objective  focused  on  defining  and  improving  upon  both 
the  officer  and  enlisted  joint  education  continuums  to  ensure  that  they  are 
in  line  with  the  development  goals  for  DOD’s  strategic  vision.  The  MECC 
identified  a  need  to  retain  many  aspects  of  the  current  officer  joint 
education  continuum,  as  well  as  a  need  to  incorporate  new  elements  into 
the  proposed  officer  joint  education  continuum  that  would  help  to  meet 
the  changing  needs  of  future  leaders  of  Joint  Force  2020.  For  example, 
the  proposed  officer  continuum  retains  the  comprehensive  approach  of 
embedding  elements  of  JPME  at  each  level  of  service-delivered 
education  with  an  emphasis  on  the  statutory  requirements  for  JPME 
Phases  I  and  II,  as  well  as  key  milestones  from  the  precommissioning 
level  of  JPME  to  the  general/flag  officer  rank.  In  addition  to  retaining  the 
existing  institutional  structure  for  providing  JPME  education,  however,  in 
its  report  the  MECC  also  emphasized  the  importance  of  self-directed, 
career-long  learning  and  development.  This  facet  of  officer  development 
is  intended  to  convey  an  expectation  that  an  individual  is  responsible  for 
his  or  her  own  education  and  development,  thereby  inculcating  a  culture 
of  lifelong  learning.  The  MECC’s  study  concluded,  on  the  other  hand,  that 
changes  to  the  enlisted  JPME  continuum — specifically,  between  the 
Senior  Enlisted  JPME  and  KEYSTONE  courses —  are  needed  to  improve 
the  depiction  of  the  Chairman’s  intent  for  enlisted  professional  military 
education.  Currently,  enlisted  personnel  who  complete  the  Senior 
Enlisted  JPME  course  at  pay  grade  E-6  or  E-7  are  not  eligible  to  attend 
the  KEYSTONE  course  until  they  attain  pay  grade  E-9,  potentially  leaving 
a  significant  interval  in  time  between  participation  in  the  two  courses. 
Accordingly,  the  MECC  recommended  dividing  the  existing  Senior 
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Enlisted  JPME  course  into  two  parts.  Both  parts  would  be  offered  prior  to 
the  KEYSTONE  course  and  reduce  the  interval  of  time  that  senior 
enlisted  servicemembers  must  wait  between  phases  of  enlisted  joint 
education.24 

The  second  objective  expanded  upon  the  first,  and  focused  on  identifying 
and  closing  gaps  in  the  current  JPME  curriculum  to  produce  joint  leaders 
with  attributes  that  are  in  line  with  Joint  Force  2020.  Specifically,  the 
MECC  refined  the  four  attributes  proposed  in  the  Chairman’s  white  paper, 
added  two  additional  attributes,  and  identified  a  set  of  subattributes 
associated  with  each  of  the  six  desired  leader  attributes,  that  the  MECC 
used  to  analyze  the  current  capabilities  of  the  department’s  joint 
education  programs.  The  refined  list  of  desired  leader  attributes  consists 
of  the  abilities  to: 

1 .  Understand  the  security  environment  and  contributions  of  all 
instruments  of  national  power, 

2.  Anticipate  and  respond  to  surprise  and  uncertainty, 

3.  Anticipate  and  recognize  change  and  lead  transitions, 

4.  Operate  on  intent  through  trust,  empowerment,  and  understanding, 

5.  Make  ethical  decisions  based  on  shared  values  of  the  profession  of 
arms,  and, 

6.  Think  critically  and  strategically  in  applying  joint  warfighting  principles 
and  concepts  to  joint  operations. 


The  MECC  then  conducted  a  gap  analysis  to  identify  and  crosswalk  the 
desired  leader  attributes  for  Joint  Force  2020  with  the  current  officer  and 
enlisted  personnel  JPME  education  continuums. 

For  officers,  the  MECC’s  analysis  showed  that,  while  the  JPME  program 
currently  addresses  the  desired  leader  attribute  outcomes  at  some  level 
and  also  meets  the  intent  of  the  existing  requirements  identified  by  the 
Skelton  Panel,  gaps  exist  where  changes  are  needed  to  meet  the 
challenges  of  Joint  Force  2020.  These  include  changes  to  teaching 
methodologies,  assessment  mechanisms,  and  other  areas  in  support  of 
the  newly-identified  desired  leader  attributes.  Table  1  summarizes  where 


24We  discuss  DOD’s  efforts  to  address  the  MECC’s  recommendations  later  in  this  report. 
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DOD  has  identified  gaps  in  its  officer  JPME  curricula  associated  with  the 
identified  list  of  desired  leader  attributes  and  subattributes. 

Table  1:  Gaps  in  Officer  Curricula  Associated  with  the  Desired  Leader  Attributes 

Joint  Force  2020  Desired  Leader  Attributes  (DLAs) 

DOD  assessment  of  the 
existence  of  gaps 

Examples  of  identified  gaps 

1 .  Ability  to  understand  the  security  environment  and 
the  contributions  of  all  instruments  of  national  power. 

• 

No  gaps  in  curriculum. 

2.  Ability  to  anticipate  and  respond  to  surprise  and 
uncertainty. 

O 

Additional  faculty  development  may  be 
required  for  instruction. 

3.  Ability  to  anticipate  and  recognize  change  and  lead 
transitions. 

€ 

Additional  scenarios  and  simulations  for 
instruction  may  be  required. 

4.  Ability  to  operate  on  intent  through  trust, 
empowerment,  and  understanding. 

O 

Some  subattributes  of  desired  leader 
attributes  are  not  covered  in  curriculum  and 
may  require  additional  technology  for 
simulations. 

5.  Ability  to  make  ethical  decisions  based  on  the  shared 
values  of  the  profession  of  arms. 

• 

No  gaps  in  curriculum. 

6.  Ability  to  think  critically  and  strategically  and  apply 
joint  warfighting  principles  and  concepts  in  joint 
operations  spanning  all  levels  of  warfare. 

• 

No  gaps  in  curriculum. 

Source:  DOD. 


Legend 

#  DOD  did  not  identify  any  gaps  in  its  JPME  curricula  to  meet  the  desired  leader  attribute,  and  no 
corrective  action  is  needed. 

€  DOD  found  potential  gaps  and  that  some  refinement  to  the  curriculum  may  be  needed. 

O  DOD  found  that  gaps  exist  in  the  JPME  curricula  to  meet  the  desired  leader  attribute,  and  that 
corrective  action  is  needed. 


For  enlisted  personnel,  the  MECC  determined  that  a  separate  and  distinct 
set  of  desired  leader  attributes  should  be  developed  for  senior  enlisted 
leaders  that  will  require  additional  study.  Officials  from  the  Joint  Staff 
initiated  this  work  on  April  8,  2013  and  assigned  this  task  to  DOD’s 
Enlisted  Military  Education  Review  Council  (EMERC).25  The  EMERC  took 
up  this  effort  at  its  August  22,  2013  meeting  and  proposed  a  list  of  six 
desired  leader  attributes  for  enlisted  personnel.  DOD  officials  provided  us 


25The  Enlisted  Military  Education  Review  Council  (EMERC)  serves  as  an  advisory  body  to 
the  Joint  Staff’s  Deputy  Director  for  Military  Education  on  enlisted  joint  education  issues. 
The  EMERC  addresses  issues  of  interest  to  the  joint  education  community,  promotes 
cooperation  and  collaboration  among  EMERC  member  institutions,  and  coordinates  joint 
education  initiatives. 
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with  the  proposed  list  of  enlisted  desired  leader  attributes  on  September 
16,  2013. 

The  MECC  made  a  total  of  21  recommendations,  which  collectively 
address  the  study’s  objectives  and  span  four  categories,  including: 

(1)  desired  leader  attributes,  subattributes,  or  educational  outcomes; 

(2)  joint  education  continuums;  (3)  lifelong  learning  and  advancements  in 
learning  technologies;  and  (4)  faculty  quality.  See  appendix  III  for  a  list  of 
the  21  recommendations.  Several  of  these  recommendations  call  for 
additional  study  by  the  National  Defense  University  and  the  services  to 
support  the  achievement  of  leader  attributes.  Other  recommendations 
emphasize  the  importance  of  strengthening  the  educational  outcomes  at 
the  primary  level  of  joint  education,  making  education  programs  more 
accessible,  and  using  prior  learning  assessments  to  tailor  education 
opportunities  to  students’  needs,  among  other  things.  One 
recommendation,  in  particular,  calls  for  undertaking  an  approach  to 
develop  a  separate  and  distinct  set  of  desired  leader  attributes  to  guide 
enlisted  joint  education.  Additionally,  the  MECC  considered  whether  to 
make  any  legislative  proposals,  but  did  not  identify  the  need  for  any 
legislative  changes  to  the  joint  education  enterprise  based  on  the  results 
of  the  review.  According  to  officials,  the  results  of  the  study  will  be  used 
ultimately  to  inform  updates  and  revisions  to  the  Officer  Professional 
Military  Education  Policy26  and  the  Enlisted  Professional  Military 
Education  Policy,27  which  are  the  primary  instructions  that  distribute  the 
policies,  procedures,  objectives,  and  responsibilities  for  professional 
military  education  and  JPME.  Our  next  section  discusses  DOD’s  plans  in 
this  area. 


26CJCSI  1 800.01  D. 
27CJCSI  1805. 01A. 
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Study’s  Methodology 
Generally  Included 
Leading  Practices,  but 
DOD  Has  Not  Yet 
Fully  Planned  for 
Follow-on  Actions 
and  Engaged  All 
Stakeholders,  or 
Considered  Potential 
Implementation  Costs 


The  methodology  DOD  used  for  its  study  of  JPME  generally  included 
leading  practices  for  evaluating  strategic  training  and  other  programs, 
such  as  reviewing  existing  literature,  using  available  data,  and  assessing 
skills  gaps.  However,  the  department  has  not  yet  fully  planned  for  follow- 
on  actions  or  fully  engaged  all  stakeholders  to  help  ensure  that  they  are 
held  accountable  for  making  progress  in  implementing  the  study’s 
recommendations.  Specifically,  the  department  has  not  taken  steps  to 
ensure  that  the  results  of  the  study  will  be  implemented  using  an 
approach  agreed  upon  by  all  stakeholders,  or  that  the  results  will  be 
implemented  in  a  timely  manner.  Further,  the  department  has  not  yet  fully 
evaluated  the  potential  costs  associated  with  the  implementation  of  the 
MECC’s  recommendations  to  provide  decision  makers  with  more 
complete  information  and  assurance  that  the  recommendations  will  be 
cost-effective. 


MECC  Reviewed  Existing  According  to  our  2012  report,  Designing  Evaluations,28  a  key  first  step  in 
Literature,  Used  Available  designing  a  program  evaluation  is  to  conduct  a  literature  review  in  order 
Data  and  Assessed  Gaps  t0  understand  the  program’s  history,  related  policies,  and  knowledge 

base.  A  review  of  the  relevant  policy  literature  can  help  focus  evaluation 
questions  on  knowledge  gaps,  identify  design  and  data  collection  options 
used  in  the  past,  and  provide  important  context  for  the  requester’s 
questions.  Further,  according  to  the  RAND  Corporation’s  Standards  for 
High-Quality  Research  and  Analysis,29  a  study  team  should  demonstrate 
an  understanding  of  other  related  studies,  which  should  be  evident  in,  for 
example,  how  a  problem  is  formulated  and  approached.  The  team  should 
also  take  particular  care  to  explain  the  ways  in  which  its  study  agrees, 
disagrees,  or  otherwise  differs  importantly  from  previous  studies. 

The  MECC’s  report  states  that  the  initial  step  in  its  review  was  to 
determine  the  JPME  enterprise’s  effectiveness  in  meeting  its  current 
requirements.  According  to  an  official,  the  MECC  reviewed  many 
documents  as  part  of  a  literature  review,  but  derived  its  findings  from  four 


28GAO,  Designing  Evaluations:  2012  Revision,  GAO-12-208G  (Washington,  D.C.:  January 
2012). 

29RAND  Corporation,  Standards  for  High-Quality  Research  and  Analysis  (Santa  Monica, 
Calif.:  November  2011). 
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specific  sources — all  issued  within  the  last  5  years — that  it  determined 
were  most  relevant  to  the  review.  Specifically,  these  sources  were: 

•  The  2010  House  Armed  Services  Committee  (HASC),  Subcommittee 
on  Oversight  and  Investigations  (O&l),  study30  which  provided  a 
detailed  assessment  of  the  state  of  JPME; 

•  A  Decade  of  War,31  conducted  by  the  Joint  and  Coalition  Operational 
Analysis  Division32  within  the  Joint  Staff  (J7); 

•  The  Ingenuity  Gap:  Officer  Management  for  the  21st  Century33;  and 

•  Keeping  the  Edge:  Revitalizing  America’s  Military  Officer  Corps.34 


According  to  the  MECC’s  report,  these  documents  collectively  provide  a 
broad  look  across  joint  education.  The  MECC  found  that  the  2010  HASC 
report,  for  example,  provided  a  detailed  assessment  of  the  state  of  JPME, 
and  concluded  that  while  the  overall  PME  system  was  basically  sound, 
some  areas  needed  improvement,  such  as  an  increased  emphasis  on 
joint,  interagency,  intergovernmental,  and  multinational  operations. 
Consistent  with  this  finding,  the  MECC  made  a  recommendation  that  the 
Joint  Professional  Military  Education  Division  review  specific  subject 
areas  for  increased  emphasis  within  joint  education  as  part  of  the  process 
of  revising  the  officer  and  enlisted  policies.  These  subject  areas  include 
cyber  warfare,  interagency  and  intergovernmental  operations,  and 
information  and  economic  instruments  of  national  power,  among  others. 
Further,  these  sources  suggested  that  responsibilities  for  junior  officers 
will  increase  and  that  joint  education  should  be  expanded  at  lower  levels. 


on 

Another  Crossroads?  Professional  Military  Education  Two  Decades  After  the  Goldwater- 
Nichols  Act  and  the  Skelton  Panel 

o1 

Joint  Staff  J-7  Joint  and  Coalition  Operational  Analysis  (JCOA),  Decade  of  War,  Volume 
I:  Enduring  Lessons  from  the  Past  Decade  of  Operations  (Suffolk,  VA:  June  15,  201 2). 

32ln  support  of  the  Chairman’s  Joint  Lessons  Learned  Program,  and  as  directed,  the  Joint 
Staff  J7  Joint  and  Coalition  Operational  Analysis  (JCOA)  Division  collects,  aggregates, 
analyzes,  and  disseminates  joint  lessons  learned  and  best  practices  across  the  range  of 
military  operations  in  order  to  enhance  joint  capabilities.  JCOA  examined  over  400 
findings,  observations,  and  best  practices  to  identify  enduring  lessons  that  can  inform 
future  joint  force  development  in  its  Decade  of  War,  Volume  I  study. 

oo 

Maren  Leed  and  David  Sokolow,  The  Ingenuity  Gap:  Officer  Management  for  the  21st 
Century  (Washington,  D.C.:  Center  for  Strategic  and  International  Studies,  2010). 

34John  A.  Nagl  and  Brian  M.  Burton,  Keeping  the  Edge:  Revitalizing  America’s  Military 
Officer  Corps  (Washington,  D.C.:  Center  for  a  New  American  Security,  2010). 
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Similarly,  the  MECC  found  that  officers  are  receiving  joint  education 
exposure  earlier  in  their  careers,  and  also  recommended  that  the  services 
strengthen  the  instruction  of  the  desired  leader  attributes  at  the  primary 
level  of  joint  education  for  junior  officers.  Additionally,  these  sources 
recognized  the  need  for  increased  emphasis  on  handling  uncertainty  and 
critical  thinking  skills.  DOD’s  study  also  focused  on  the  ability  to  anticipate 
and  respond  to  surprise  and  uncertainty,  and  critical  thinking  in  joint 
operations. 

RAND’s  Standards 35  report  further  indicates  that  the  data  and  information 
used  for  a  study  should  be  the  best  available  and  that  a  research  team 
should  indicate  limitations  in  the  quality  of  the  available  data.  As  part  of  its 
second  objective,  the  MECC  analyzed  data  from  joint  accreditation  visits, 
that  provided  broad  areas  of  assessment,  and  institutional  survey  and 
outcome  assessments,  that  provided  more  detailed  observations  related 
to  curricula.  More  specifically,  the  MECC  conducted  a  review  of  the 
Process  for  the  Accreditation  of  Joint  Education  (PAJE)  reaffirmation 
results  and  institutional  self-studies  for  the  past  10  years.36  DOD  uses 
these  studies  to  document  compliance  with  the  Officer  Professional 
Military  Education  Policy  (OPMEP)  standards  and  identify  areas  for 
improvement.  These  reviews  have  not  identified  systemic  problems  or 
required  changes  of  PME  over  the  past  10  years,  and  have  concluded 
that  PME  institutions  are  successful  in  meeting  policy  and  curricula 
standards,  as  well  as  the  intent  of  the  Skelton  Report.  The  MECC  also 
obtained  and  analyzed  existing  survey  data 37 — from  various  JPME 
institutions — of  students,  graduates,  and  supervisors  of  graduates  as  an 
indirect  assessment  of  the  programs  to  identify  common  areas  and  trends 
in  JPME,  as  well  as  outcome  assessments  such  as  exams,  quizzes,  oral 
presentations,  among  other  things,  from  the  past  2  years  as  a  more  direct 
measurement  of  student  learning.  Consistent  with  the  aforementioned 
RAND  Standards  report,  while  the  MECC  did  leverage  existing  data,  the 


35 Standards  for  High-Quality  Research  and  Analysis. 

36 According  to  the  MECC’s  report,  the  peer-review  PAJE  process  is  conducted  every  6 
years  and  the  reports  provide  a  continuous,  systemic  examination  of  the  health  of  the 
JPME  system.  The  institutional  self-studies  are  detailed  assessments  and  form  the 
foundation  for  the  PAJE  on-site  visits. 

37The  MECC  report  states  that  because  the  vast  majority  of  data  concerning  joint 
education  have  been  focused  on  the  statutorily  mandated  levels  of  JPME  for  officers 
(JPME  Phase  1 ,  JPME  Phase  II,  and  CAPSTONE),  the  data  analysis  portion  of  the 
second  objective  focuses  solely  on  those  programs. 
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MECC’s  report  also  presented  limitations  of  the  existing  survey  data  that 
it  used  for  the  study,  stemming  from  variations  in  survey  questions  and 
measures  that  the  separate  JPME  institutions  used  to  conduct  their  own 
surveys. 

Regarding  gap  assessments,  our  2004  Guide  for  Assessing  Strategic 
Training  and  Development  Efforts  in  the  Federal  Government 38  highlights 
the  need  for  determining  the  skills  and  competencies  necessary  to  meet 
current  and  future  challenges,  and  to  assess  any  skill  and  competency 
gaps.  According  to  the  MECC’s  report,  to  determine  the  ability  of  joint 
education  to  meet  the  Joint  Force  2020  joint  leader  development 
requirements,  the  MECC  conducted  a  three-phase  gap  analysis.  These 
three  phases  included:  (1)  refinement  of  the  desired  leader  attributes, 
which  included  breaking  these  attributes  down  into  subattributes; 

(2)  analysis  of  data  provided  by  27  officer  and  enlisted  programs  across 
the  JPME  system  to  identify  gaps  in  the  JPME  program;39  and  (3)  gap 
analysis  and  documentation  to  break  down  the  data  into  cohorts  and  rank 
the  desired  leader  attributes.  Specifically,  27  officer  and  enlisted 
programs,  which  included  the  services’  staff  and  war  colleges  and 
National  Defense  University,  among  others,  were  asked  to  assess  the 
current  capabilities  of  their  joint  education  programs  to  achieve  Joint 
Force  2020  outcomes  by  addressing  the  following  three  questions: 

•  Is  the  curriculum  effective,  that  is,  do  we  teach  it? 

•  Are  current  instructional  methods  effective,  that  is,  how  well  do  we 
teach  and  deliver  it? 

•  Are  we  achieving  the  educational  outcomes,  that  is,  can  we  measure 
it? 

According  to  its  report,  the  MECC  relied  on  the  professional  opinion  of  the 
academic  deans  of  the  JPME  schools  and  directors  of  other  joint 
education  programs  to  make  determinations  about  the  effectiveness  of 
their  officer  and  senior  enlisted  programs  based  on  the  35  subattributes 
identified  by  the  MECC.  To  conduct  the  gap  analysis,  the  MECC  analyzed 


oo 

GAO,  Human  Capital:  A  Guide  for  Assessing  Strategic  Training  and  Development 
Efforts  in  the  Federal  Government,  GAO-04-546G  (Washington,  D.C.:  March  2004). 

39The  27  officer  and  enlisted  programs  surveyed  as  part  of  the  gap  analysis  data 
collection  included  two  precommissioning  level,  ten  intermediate  level,  eight  senior  level 
and  CAPSTONE  programs,  and  six  senior  enlisted  programs.  According  to  the  MECC’s 
report,  data  were  requested  from  primary  schools  but  omitted  from  analysis  due  to  time 
constraints. 
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the  945  responses  it  received  from  the  27  programs  and  used  a  stoplight 
color  coding  scheme  to  summarize  by  cohort  group  each  program’s 
assessment  of  its  current  capabilities  to  achieve  the  Joint  Force  2020 
outcomes.40  The  gap  analysis  results  were  then  coded,  scored  and 
ranked  by  officer  cohort  group  (precommissioning,  intermediate,  senior, 
CAPSTONE)  using  four  categories  to  describe  the  program’s  current 
capabilities.  In  turn,  this  gap  analysis  helped  inform  the  MECC’s  21 
recommendations.  While  the  MECC  conducted  a  similar  gap  analysis  for 
its  senior  enlisted  JPME  programs,  the  results  of  that  analysis  led  the 
MECC  to  conclude  that  the  department  needed  to  further  reconsider  and 
redefine  the  desired  leader  attributes  for  enlisted  personnel. 


MECC  Has  Not  Yet  Fully 
Developed  Timeframes 
and  Coordinated  with  All 
Stakeholders,  nor 
Assessed  Costs  for  Follow- 
on  Action 


The  MECC  has  not  yet  fully  developed  timeframes  and  coordinated  with 
all  stakeholders,  nor  has  it  assessed  the  costs  associated  with 
implementing  any  of  the  study’s  recommendations.  As  a  result,  it  is  not 
clear  how  the  department  will  obtain  concurrence  among  its  stakeholders 
regarding  an  approach  for  moving  forward,  or  how  the  department  will 
hold  its  stakeholders  accountable  for  implementing  change.  Furthermore, 
the  department  will  not  know  whether  the  implementation  of  its 
recommendations  will  be  cost  effective. 


We  have  previously  found41  that  project  planning  is  the  basis  for 
controlling  and  managing  project  performance,  including  managing  the 
relationship  between  cost  and  time.  Our  2004  Training  Guide42  highlights 
the  need  for  agencies  to  develop  a  formal  process  to  help  ensure  that 
strategic  and  tactical  changes  are  promptly  incorporated  in  training  and 
development  efforts.  Specifically,  that  report  advises  that  agencies 
develop  plans  that  describe  or  outline  the  way  in  which  the  agency 
intends  to  incorporate  strategic  and  tactical  changes  into  its  training  and 
developmental  efforts.  Our  guide  also  highlights  that  including  important 
agency  stakeholders  in  the  process  can  contribute  to  an  open  and 


40We  did  not  independently  assess  the  responses  of  the  27  programs  or  the  MECC’s 
determinations  of  any  gaps  as  part  of  our  review.  For  the  purposes  of  our  review,  we  have 
found  that  conducting  a  gap  analysis  is  a  leading  practice  and  reviewed  the  MECC’s 
approach  accordingly. 

41  GAO,  GAO  Schedule  Assessment  Guide:  Best  Practices  for  Project  Schedules, 

GAO-1 2-1 20G  (Washington,  D.C.:  May  2012). 

42GAO-04-546G. 
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continuous  exchange  of  ideas  and  information,  particularly  when  it  comes 
to  ensuring  that  strategic  and  tactical  changes  are  promptly  incorporated 
into  training  and  developmental  efforts. 

According  to  the  July  16,  2012  memorandum  that  directed  the  MECC 
study,  the  March  1 , 201 3  deadline  for  the  results  of  the  study  was 
intended  to  help  shape  the  fall  2013  academic  year.  The  MECC 
completed  its  report  on  June  24,  2013,  and  provided  us  with  a  copy  of  the 
final  report  on  July  1,  2013.  Subsequently,  officials  provided  us  with  an 
informational  paper  that  identifies  an  office  of  primary  responsibility  and 
outlines  details  on  the  status  of  the  recommendations.  However,  this 
paper  does  not  fully  lay  out  DOD’s  planned  actions  in  a  transparent 
manner.  For  nine  of  the  recommendations,  the  document  identifies 
potential  target  dates  for  completion,  one  of  which  is  to  be  completed  later 
in  2013,  and  eight  of  which  are  to  be  completed  not  later  than  September 
2014.  For  the  remaining  12  recommendations,  DOD  identified  a  target 
date  for  completion  as  either  “not  applicable”  or  “to  be  determined” 
because  most  of  those  recommendations  fall  under  the  purview  of  the 
services,  the  services’  schools,  or  National  Defense  University. 
Accordingly,  officials  states  that  consideration  or  implementation  of  any 
recommendations  that  fall  under  the  purview  of  these  stakeholders  is  at 
the  discretion  of  those  organizations  and,  according  to  officials,  the 
department  cannot  identify  or  impose  timeframes  for  implementing  those 
recommendations.  We  recognize  that  the  implementation  of  several  of  the 
recommendations  fall  under  the  purview  of  the  services,  the  services’ 
schools,  or  National  Defense  University.  However,  the  MECC  has  not  yet 
reached  out  to  these  entities  to  identify  estimated  target  dates  for 
implementation  to  help  guide  these  efforts.  Further,  the  document  does 
not  identify  interim  milestones  for  specific  actions  between  the  date  of  the 
report  and  the  target  completion  dates.  The  document  also  states  that 
completion  or  implementation  of  a  number  of  the  recommendations  is 
contingent  on  revisions  to  the  Enlisted  Professional  Military  Education 
Policy  or  the  Officer  Professional  Military  Education  Policy,  which  officials 
estimated  may  take  12-18  months. 

Officials  did  tell  us  that  the  Joint  Staff’s  Joint  Professional  Military 
Education  Division  staff  and  members  of  the  MECC  are  currently  in  the 
beginning  stages  of  incorporating  the  changes  into  policy  and  that  the 
individual  schools  may  begin  to  incorporate  changes  resulting  from  the 
study  at  any  time.  However,  at  the  time  of  our  review,  the  department’s 
process  for  monitoring  the  implementation  of  the  MECC’s 
recommendations  was  unclear.  Without  established  milestones  and 
timeframes  for  taking  follow-on  action,  and  mechanisms  for  coordinating 
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with  all  key  officials  and  stakeholders  and  holding  them  accountable, 

DOD  may  not  be  reasonably  assured  that  the  study’s  recommendations 
will  be  implemented  across  the  department  or  that  its  JPME  program  will 
achieve  the  potential  benefits  resulting  from  these  recommendations. 

In  addition,  the  department  did  not  include  a  requirement  that  the  MECC 
assess  the  costs  associated  with  the  JPME  program  or  any 
recommendations  resulting  from  the  study.  The  MECC’s  report 
acknowledges  that  the  resource-limited  environment  facing  the 
Department  of  Defense  will  make  it  difficult  to  sustain  current  practices, 
such  as  maintaining  small  class  sizes  and  small  student-to-faculty  ratios, 
at  the  intermediate  and  senior  courses  or  to  consider  additional 
requirements  for  the  lower-level  courses.  The  report  states  that 
leveraging  new  learning  approaches,  such  as  increased  distance  learning 
options,  will  not  provide  an  inexpensive  or  free  solution  for  increased  joint 
education  requirements.43  However,  the  MECC  did  not  analyze  or 
consider  either  the  costs  associated  with  implementing  the  study’s 
recommendations  or  efficiencies  to  be  derived  from  implementing  the 
study’s  recommendations.  Of  the  MECC’s  21  recommendations,  the 
MECC  identified  five  recommendations  requiring  further  study  and  other 
possible  investments,  which  suggests  a  potential  need  for  additional 
funding  resources  to  conduct  those  studies  and  implement  any  findings. 
For  example,  the  report  included  recommendations  to  (1)  conduct  a  study 
to  identify  and  evaluate  potential  educational  tools  to  achieve  certain 
desired  leader  attributes,  and  (2)  consider  and  explore  opportunities  to 
incentivize  and  reward  lifelong  learning.  The  costs  associated  with  these 
recommendations  are  unclear  and  officials  responsible  for  the  study  could 
not  provide  us  with  cost  estimates. 

Prior  studies  of  professional  military  education  also  did  not  or  were  unable 
to  fully  identify  the  costs  associated  with  the  overall  JPME  program,  or 
with  individual  JPME  programs.  The  Skelton  Panel,  for  example,  inquired 
into  the  cost  per  student  at  each  school.  The  Office  of  the  Secretary  of 
Defense  provided  the  panel  with  raw  data  produced  with  different 
methodologies  by  service,  and  sometimes  by  school,  which  resulted  in 


43The  MECC  report  states  that  the  existing  JPME  process  implements  an  adult  learning 
model  that  comes  at  the  high  price  of  small  class  sizes  and  small  student-to-faculty  ratios. 
Most  courses  for  junior  officers  and  enlisted  personnel  do  not  have  the  resources  to 
support  this  model;  thus  adding  more  joint  education  in  officer  and  enlisted  policy  could 
pose  challenges  at  all  levels. 
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widely  varying  costs  for  roughly  similar  programs.  More  recently,  the 
House  Armed  Services  Subcommittee  for  Oversight  and  Investigations44 
made  its  own  effort  to  ascertain  whether  a  uniform  cost  accounting 
system  existed  for  DOD’s  professional  military  education  system  45  The 
Department  provided  cost-per-student  figures  with  standardized  criteria. 
Comparative  figures  were  no  longer  characterized  by  such  enormous 
variations,  but  there  were  still  a  number  of  unexplained  differences  in  the 
data  provided  by  the  Office  of  the  Secretary  of  Defense. 

Our  2004  Training  Guide 46  states  that  agencies  should  strategically  target 
training  and  development  investments  to  help  ensure  that  resources  are 
not  wasted  on  efforts  that  are  irrelevant,  duplicative,  or  ineffective.  Our 
Training  Guide  further  recommends  that  agencies  should  consider  the 
appropriate  level  of  investment  and  prioritize  funding  so  that  the  most 
important  training  needs  are  addressed  first.  We  have  previously 
concluded47  that  leading  practices  such  as  accounting  for  program 
resources  enables  managers  to  better  manage  existing  resources  and 
plan  for  future  programmatic  needs.  Further,  accounting  for  program 
resources  is  becoming  increasingly  important  for  decision  makers  within 
the  department  who  must  make  difficult  trade-off  decisions  in  a 
sequestration  budget  environment. 

According  to  officials,  the  MECC  omitted  any  and  all  cost  elements  from 
the  study  so  as  to  not  limit  their  assessment  of  the  department’s  needs  for 
educating  its  leaders  for  Joint  Force  2020  based  on  resources.  Officials 
told  us  that  they  did  not  want  to  eliminate  options  for  change  based  on  the 
cost  associated  with  them.  However,  by  not  assessing  the  cost  of  the 
MECC’s  recommendations,  as  well  as  any  efficiencies  that  may  be 
achieved  through  their  implementation,  DOD  may  not  be  in  a  position  to 
know  if  it  has  the  resources  to  implement  them  or  whether  efficiencies  will 
be  achieved,  and,  without  plans  to  assess  cost  in  the  near  term  as  part  of 
continued  efforts  to  implement  the  results  of  the  JPME  study,  decision 


44 Another  Crossroads?  Professional  Military  Education  Two  Decades  After  the  Goldwater- 
Nichols  Act  of  1 986  and  the  Skelton  Panel  Study  of  1 989. 

45The  House  Armed  Services  Committee’s  study  was  focused  on  professional  military 
education  as  a  whole,  of  which  joint  professional  military  education  is  a  subset. 

46GAO-04-546G. 

47GAO,  Environmental  Justice:  EPA  Needs  to  Take  Additional  Actions  to  Help  Ensure 
Effective  Implementation ,  GAO-12-77  (Washington,  D.C.:  Oct.  6,  2011). 
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makers  may  not  know  if  implementing  their  recommendations  will  be  cost 
effective. 


Conclusions 


The  experiences  of  the  last  12  years  of  operations  in  Iraq  and 
Afghanistan  have  underscored  the  importance  of  U.S.  military  officers’ 
ability  to  work  jointly  across  the  services  and  reinforced  the  need  for  an 
effective  joint  military  education  program.  DOD’s  recently  completed 
study  of  JPME  is  an  important  step  forward  as  DOD  considers  how  to 
adapt  its  joint  military  education  program  to  reach  the  goals  it  laid  forth  for 
Joint  Force  2020.  Nonetheless,  some  questions  remain  about  how  well  its 
findings  can  be  applied  to  the  task  of  revising  the  department’s  officer  and 
enlisted  JPME  policies.  Specifically,  ensuring  transparency  and  taking 
certain  actions  to  ensure  this  study’s  findings  and  recommendations  can 
be  utilized  effectively  is  important.  For  instance,  without  establishing 
milestones  and  timeframes  for  conducting  follow-on  actions,  and  involving 
necessary  stakeholders  and  developing  an  implementation  approach  that 
provides  for  accountability,  the  department  may  not  be  assured  that  the 
study’s  findings  and  recommendations  will  be  incorporated  into  the  JPME 
program  at  institutions  across  the  department  and  the  services  in  a  timely 
manner.  Additionally,  without  reliable  information  on  the  costs  of 
implementing  the  study’s  recommendations,  decision  makers  could  be 
hindered  in  determining  the  most  efficient  allocation  of  departmental 
resources  for  JPME.  DOD’s  JPME  program  will  continue  to  play  a  role  in 
the  future  development  of  departmental  leaders,  and  acting  to  help 
ensure  the  findings  and  recommendations  of  this  and  any  future  studies 
are  both  utilized  and  cost-effective  could  allow  the  department  to  make 
the  best  use  of  the  resources  it  devotes  to  this  program. 


RprnmmpnHatinns  for  9u'de  implementation  of  actions  DOD  identified  in  its  study  on 

JPME,  we  recommend  that  the  Chairman  of  the  Joint  Chiefs  of  Staff 
Executive  Action  direct  the  Director  for  Joint  Force  Development  to  take  the  following  two 

actions: 

•  establish  well-defined  timeframes  for  conducting  follow-on  actions, 
coordinate  with  all  stakeholders,  and  identify  key  officials 
responsible  for  implementing  the  study’s  recommendations  to  help 
ensure  the  usefulness,  timeliness,  and  implementation  of  any 
actions  DOD  takes  in  response  to  the  findings  and 
recommendations  contained  in  its  study,  and 
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•  assess  the  costs  of  implementing  recommendations  made  and 
efficiencies  to  be  derived  from  the  recommendations  in  order  to 
implement  DOD’s  recommendations  in  a  cost-effective  manner. 

Agency  Comments 

In  written  comments  on  a  draft  of  this  report,  DOD  concurred  with  our  two 
recommendations  to  guide  implementation  of  the  actions  that  DOD 
identified  in  its  review  of  joint  education.  DOD’s  comments  are  reprinted 
in  appendix  IV.  DOD  also  provided  technical  comments  on  the  draft 
report,  which  we  incorporated  as  appropriate. 

Regarding  our  first  recommendation  that  DOD  establish  well-defined 
timeframes  for  conducting  follow-on  actions,  coordinate  with  all 
stakeholders,  and  identify  key  officials  responsible  for  implementing  the 
study’s  recommendations,  DOD  stated  that,  as  the  Joint  Staff  pursues 
implementation  of  the  recommendations  in  its  review,  it  will  establish 
timelines,  identify  offices  of  primary  responsibility,  and  coordinate  with 
stakeholders  as  appropriate.  DOD  also  noted  that  efforts  are  already 
underway  to  update  the  Chairman  of  the  Joint  Chief  of  Staff’s  professional 
military  education  policies  to  ensure  that  these  policy  directives  are 
consistent  with  the  results  and  recommendations  of  the  recent  study. 

Regarding  our  second  recommendation  that  DOD  assess  the  costs  of 
implementing  recommendations  made  and  efficiencies  to  be  derived  from 
the  recommendations,  the  department  again  concurred,  stating  that, 
although  assessing  the  costs  of  its  recommendations  is  beyond  the 
original  scope  and  purpose  in  this  study,  it  will  consider  costs  and 
efficiencies  prior  to  moving  forward  with  the  implementation  of  any 
recommendations.  It  also  stated  that  a  number  of  recommendations  will 
be  implemented  through  policy  changes  which  are  cost  neutral  Further,  it 
stated  that  the  services  will  also  consider  such  costs  and  efficiencies  for 
recommendations  under  their  purview. 

We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees,  the  Chairman  of  the  Joint  Chiefs  of  Staff,  and  the  Directorate 
for  Joint  Force  Development.  In  addition,  this  report  will  also  be  available 
at  no  charge  on  the  GAO  Web  site  at  http://gao.gov. 
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If  you  or  your  staff  have  any  questions  regarding  this  report,  please 
contact  me  at  (202)  512-3604  or  farrellb@gao.gov.  Contact  points  for  our 
Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found  on 
the  last  page  of  this  report.  GAO  staff  who  made  major  contributions  to 
this  report  are  listed  in  appendix  V. 


Brenda  S.  Farrell 
Director 

Defense  Capabilities  and  Management 
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List  of  Committees 


The  Honorable  Carl  Levin 
Chairman 

The  Honorable  James  M.  Inhofe 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Howard  P.  “Buck”  McKeon 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 
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Appendix  I:  Scope  and  Methodology 


To  identify  the  purpose  of  the  Department  of  Defense’s  (DOD)  study  of 
the  Joint  Professional  Military  Education  (JPME)  program,  we  reviewed 
relevant  DOD  guidance  and  other  documents  related  to  the  study, 
including  the  final  report  dated  June  24,  2013,  and  met  with 
knowledgeable  officials.  We  reviewed  the  Chairman  of  the  Joint  Chiefs  of 
Staffs  July  16,  2012  white  paper  that  called  for  and  outlined  the  general 
focus  of  the  study.  We  also  reviewed  the  Director  for  Joint  Force 
Development’s  (J7)  memorandum  that  was  issued  with  the  white  paper 
and  clarified  the  overall  intent  of  the  study  and  specific  questions  to  help 
frame  the  study,  and  directed  responsibility  of  the  study  to  the  Military 
Education  Coordination  Council  (MECC).  We  analyzed  the  MECC’s 
meeting  minutes  and  interim  briefing  report,  which  provided  insight  into 
how  the  MECC  addressed  the  questions  posed  in  the  Director’s 
memorandum,  as  its  approach  differed  somewhat  from  the  framework 
provided  in  the  memorandum.  We  then  compared  the  aforementioned 
guidance  documents  and  interim  report  findings  to  the  final  report  to 
assess  consistency.  In  addition,  we  met  with  knowledgeable  officials  from 
the  Joint  Professional  Military  Education  Division  within  DOD’s  Joint 
Staff’s  Directorate  for  Joint  Force  Development  who,  in  conjunction  with 
the  MECC,  were  responsible  for  guiding  DOD’s  effort.  We  also  met  with 
the  officers  of  primary  responsibility  and/or  team  leads  responsible  for 
carrying  out  the  various  elements  of  the  study.  These  meetings  helped  to 
ensure  our  understanding  of  the  purpose  and  objectives  of  the  study.  We 
reviewed  the  final  report  section  that  focused  primarily  on  the  study’s 
findings  and  recommendations  for  change  to  gain  an  understanding  of  the 
MECC’s  findings  and  recommendations.  Further,  we  met  with 
knowledgeable  officials  within  the  Directorate  for  Joint  Force 
Development  and  the  team  lead  for  the  findings  and  recommendations 
portion  of  the  study  to  gain  clarification  and  insight  into  the  major 
takeaways  from  the  study,  areas  in  need  of  change,  and  any  plans  for  the 
way  ahead. 

To  assess  the  methodology  DOD  used  to  conduct  the  joint  professional 
military  education  study  and  its  planning  for  follow-on  actions,  we 
reviewed  the  MECC’s  final  report  and  supporting  documents.  We 
interviewed  DOD  officials  from  the  Joint  Professional  Military  Education 
Division  within  the  Directorate  for  Joint  Force  Development  and  officials 
from  across  DOD  who  were  identified  as  officers  of  primary  responsibility 
or  team  leads  for  specific  study  tasks  associated  with  the  study’s  two 
objectives.  We  reviewed  relevant  reports  and  guidance  by  GAO;  the 
Office  of  Management  and  Budget;  the  Office  of  Personnel  Management; 
and  private  sector  organizations  such  as  the  RAND  Corporation,  among 
others,  to  identify  select  leading  practices  for  evaluating  training, 
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Appendix  I:  Scope  and  Methodology 


educational,  and  other  programs.  The  following  list  includes  select 
sources  we  consulted  for  leading  practices,  in  addition  to  those  listed 
throughout  the  report,  and  against  which  we  assessed  DOD’s  JPME 
study: 

•  GAO,  GAO  Schedule  Assessment  Guide:  Best  Practices  for  Project 
Schedules,  GAO-12-120G  (Washington,  D.C.:  May  2012), 

•  GAO,  Designing  Evaluations:  2012  Revision,  GAO-1 2-208G 
(Washington,  D.C.:  January  2012), 

•  RAND  Corporation,  Standards  for  High-Quality  Research  and 
Analysis  (Santa  Monica,  Calif.:  November  201 1), 

•  The  Office  of  Personnel  Management,  Training  Evaluation  Field 
Guide  (Washington,  D.C.:  January  2011), 

•  GAO,  Government  Auditing  Standards:  2007  Revision,  GAO-07-1 62G 
(Washington,  D.C.:  January  2007), 

•  GAO,  Human  Capital:  A  Guide  for  Assessing  Strategic  T raining  and 
Development  Efforts  in  the  Federal  Government,  GAO-04-546G 
(Washington,  D.C.:  March  2004), 

•  The  Office  of  Management  and  Budget,  Circular  A-94,  Guidelines  and 
Discount  Rates  for  Benefit-Cost  Analysis  of  Federal  Programs 
(Washington,  D.C.:  Oct.  29,  1992),  and 

•  GAO,  Designing  Evaluations,  GAO/PEMD-IO.1.4  (Washington,  D.C.: 
March  1991). 

We  identified  key  practices  contained  in  these  documents  for  evaluating 
educational  programs  and  information  that  were  applicable  for  the 
purposes  of  this  engagement.  We  then  compared  DOD’s  methodology 
with  these  select  leading  practices  for  program  evaluation.  We  also 
reviewed  documents  provided  to  us  and  identified  by  DOD  officials  as 
project  plans  for  the  study,  as  well  as  documentation  of  the  department’s 
intended  follow-on  actions. 

We  conducted  this  performance  audit  from  March  through  October  2013 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 
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Appendix  II:  Officer  Joint  Professional 
Military  Education  (JPME)  Institutions 


Figure  3:  Map  of  Officer  JPME  Institutions 
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Department  of 
Defense  Institutions 


National  Defense  University  (CAPSTONE,  PINNACLE),  Washington, 
DC 

Eisenhower  School  for  National  Security  and  Resource  Management, 
Washington,  DC 

National  War  College,  Washington,  DC 
National  intelligence  University,  Washington,  DC 
Joint  Forces  Staff  College,  Norfolk,  VA 
Joint  Combined  Warfighting  School,  Norfolk,  VA 
Joint  Advanced  Warfighting  School,  Norfolk,  VA 
Advanced  JPME  Reserve  Component,  Norfolk,  VA 


Page  30 


GAO-14-29  DOD's  2013  JPME  Study 


Appendix  II:  Officer  Joint  Professional  Military 

Education  (JPME)  Institutions 

United  States  Army 

•  Army  War  College,  Carlisle,  PA 

•  United  States  Army  Command  and  General  Staff  School,  Fort 
Leavenworth,  KS 

United  States  Air 
Force 

•  Air  War  College,  Maxwell  Air  Force  Base,  Montgomery,  AL 

•  Air  Command  and  Staff  College,  Maxwell  Air  Force  Base, 

Montgomery,  AL 

United  States  Navy 

•  Naval  War  College,  College  of  Naval  Warfare  and  College  of  Naval 
Command  and  Staff,  Newport,  Rl 

United  States  Marine 
Corps 

•  United  States  Marine  Corps  War  College,  Quantico,  VA 

•  United  States  Marine  Corps  Command  and  Staff  College,  Quantico, 

VA 
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Appendix  III:  Recommendations  Made  by  the 
Military  Education  Coordination  Council  (MECC) 
in  Its  Study  of  Joint  Professional  Military 
Education 


The  MECC  made  a  total  of  21  recommendations,  which  collectively 
address  the  study’s  two  objectives  and  span  four  categories,  including: 
(1)  desired  leader  attributes,  or  educational  outcomes;  (2)  joint  education 
continuums;  (3)  lifelong  learning  and  advancements  in  learning 
technologies;  and  (4)  faculty  quality.  Several  of  these  recommendations 
call  for  additional  study  by  the  National  Defense  University  and  the 
services  to  support  the  achievement  of  leader  attributes.  Other 
recommendations  emphasize  the  importance  of  strengthening  the 
educational  outcomes  at  the  primary  level  of  joint  education,  making 
education  programs  more  accessible,  and  using  prior  learning 
assessments  to  tailor  education  opportunities  to  students’  needs,  among 
other  things.  Table  2  lists  the  recommendations  by  category. 


Table  2:  Military  Education  Coordination  Council  (MECC)  Recommendations  by  Category 


Recommendation  Category  Recommendation 

Desired  leader  attributes/  subattributes/  The  MECC  should  adopt  the  desired  leader  attributes  as  guideposts  for  officer  joint 
educational  outcomes  education. 

A  “whiteboard”  approach  should  be  conducted  to  develop  a  separate  and  distinct  set 
of  desired  leader  attributes  to  guide  enlisted  joint  education. 

Develop/refine  appropriate  educational  outcomes  to  support  achievement  of  the 
desired  leader  attributes  across  a  career  via  the  Officer  Professional  Military 
Education  Policy  (OPMEPj/Enlisted  Professional  Military  Education  Policy  (EPMEP) 
revision  processes. 

National  Defense  University  and  the  services  should  conduct  further  study  to 
evaluate  potential  educational  tools  (including  online  learning,  gaming,  and 
simulation  technologies)  that  are  available  to  support  achieving  the  desired  leader 
attributes’  educational  outcomes,  particularly  for  specific  desired  leader  attributes. 

Review  specific  subject  areas  for  increased  emphasis  within  joint  education  via  the 
OPMEP  and  EPMEP  revision  processes,  including  cyber  warfare,  cultural 
considerations  in  planning,  interagency  and  intergovernmental  operations, 
information  and  economic  instruments  of  national  power,  writing  with  precision, 
operations  with  private  entities,  and  professional  ethics. 

The  joint  training  community  should  conduct/continue  efforts  aimed  at  achieving  the 
desired  leader  attributes. 

Joint  functional  communities  should  incorporate  the  desired  leader  attributes  into 
their  education  and  training  programs  as  appropriate. 

Joint  education  continuums  Determine  the  appropriate  weighting  of  emphasis  of  the  desired  leader  attributes 

along  the  education  continuums  via  the  OPMEP/EPMEP  revision  processes. 

Strengthen  the  educational  outcomes  at  the  primary  level  of  joint  education  for  junior 
officers  within  the  OPMEP  revision  process  to  enhance  achievement  of  the  desired 
leader  attributes  (while  retaining  balance  with  service  primary  level  educational 
requirements). 

Explore  the  use  of  prior  learning  assessments  to  aid  in  tailoring  education 
opportunities. 
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Appendix  III:  Recommendations  Made  by  the 
Military  Education  Coordination  Council 
(MECC)  in  Its  Study  of  Joint  Professional 
Military  Education 


Recommendation  Category  Recommendation 

National  Defense  University  should  explore  potential  opportunities  for  making 
increased  joint  education  content  available  for  junior  officers  via  distance  learning 
capabilities. 

Services  should  explore  earlier  determination  of  follow-on  assignments  from 
education  to  allow  tailoring  of  educational  opportunities,  recognizing  that  experience 
is  critical  to  fully  develop  the  desired  learning  attributes  in  service  members. 

National  Defense  University  should  explore  breaking  the  senior  enlisted  JPME 
course  into  two  phases,  one  to  serve  the  E-6/  E-7  community  and  a  second  to  serve 
the  E-8/E-9  community. 

Revise  the  EPMEP  to  establish  a  fourth  level  of  enlisted  JPME:  Command  Senior 
Enlisted  Leaders.  KEYSTONE  remains  aligned  with  the  enlisted  PME  “executive- 
level”  for  specific  command  senior  enlisted  leader  positions. 

National  Defense  University  should  conduct  a  study  to  explore  opportunities  to 
implement  elements  of  lifelong  learning  in  support  of  joint  education.  The  study 
should  examine: 

Current  service  efforts  toward  establishing  lifelong  learning  capabilities. 

Best  lifelong  learning  practices  within  the  civilian  academic  community. 

Incorporating  the  development  of  lifelong  learning  skills  in  education  program 
curricula. 

Opportunities  to  leverage  advanced  education  technologies  to  support  lifelong 
learning  including  potentially  partnering  with  the  Advanced  Distributed  Learning 
Initiative. 

National  Defense  University  and  the  services  should  explore  opportunities  to  make 
content  from  their  education  programs  available. 

Services  should  explore  opportunities  to  incentivize  and  reward  lifelong  learning. 

Faculty  quality  Services  should  establish  procedures/policies  for  special  selection  of  military 

personnel  for  joint  education  faculty  duty  and  to  recognize/reward  faculty  duty. 

Services  and  National  Defense  University  should  review  civilian  hiring 
practices/policies  to  ensure  the  highest  level  of  quality  and  proper  breadth  of  subject- 
matter  expertise  within  their  faculties,  with  an  eye  on  whole-of-government 
approaches  in  today’s  security  environment. 

Services  and  National  Defense  University  tailor  faculty  development  programs  to 
ensure  members  can  educate  to  the  desired  leader  attributes  and  sustain  current, 
relevant,  and  rigorous  curricula. 

In  light  of  the  new  desired  leader  attributes  and  lifelong  learning  strategy,  current 
standards  that  address  the  requirements  for  faculty  development  should  be 
reexamined  within  the  OPMEP  revision  process. 


Lifelong  learning  and  advancements  in 
learning  technologies 


Source:  DOD 
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Appendix  IV:  Comments  from  the 
Department  of  Defense 


THE  JOINT  STAFF 

WASHINGTON,  DC 

4  October  2013 


Ms.  Brenda  Farrell 

Director,  Defense  Capabilities  and  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington,  DC  20548 

Dear  Ms.  Farrell: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  U.S.  Government  Accountability 
Office  (GAO)  Draft  Report,  GAO-14-29,  “Joint  Military  Education:  Actions  Needed  to 
Implement  DoD  Recommendations  for  Enhancing  Leadership  Development,”  dated 
18  September  2013  (GAO  351810).  I  appreciate  GAO’s  effort  in  examining  the  analytical 
approach  used  by  the  Military  Education  Coordination  Council  (MECC)  in  conducting  our 
Review  of  Joint  Education  and  the  opportunity  to  provide  this  response  on  behalf  of  the  Chairman 
of  the  Joint  Chiefs  of  Staff. 

The  purpose  of  this  Chairman-directed,  internal  review  was  to  ensure  our  joint  education 
enterprise  will  enable  us  to  develop  agile  and  adaptive  leaders  with  the  requisite  values,  strategic 
vision,  and  critical  thinking  skills  to  keep  pace  with  the  changing  security  environment.  From  our 
perspective,  the  MECC  accomplished  the  specific  tasks  as  assigned  and  met  the  Chairman’s  intent 
for  the  review.  We  believe  much  of  the  draft  GAO  report  validates  that  perspective,  specifically 
stating  the  MECC’s  methodology  “generally  included  leading  practices  for  evaluating  strategic 
training  and  other  programs,  such  as  reviewing  existing  literature,  using  available  data,  and 
assessing  skills  gaps.” 

We  note  that  within  the  report  the  GAO  makes  two  recommendations  concerning 
timeframes  for  implementation  of  Review  of  Joint  Education  recommendations  and  assessing  any 
associated  costs  of  implementing  recommendations.  As  we  determine  which  of  the  Review  of 
Joint  Education  recommendations  will  be  pursued,  the  Joint  Staff  will  develop  appropriate 
timelines  for  and  consider  costs  of  implementation  as  necessary.  Please  see  the  enclosure  for  our 
detailed  response  to  the  two  GAO  recommendations. 

Sincerely, 


Enclosure 


FREDERICK  S.  RUDESHEIM 


Major  General,  USA 
Vice  Director,  Joint  Staff 
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Appendix  IV:  Comments  from  the  Department 
of  Defense 


GAO  DRAFT  REPORT  DATED  SEPTEMBER  18, 2013 
GAO-14-29  (GAO  CODE  351810) 

“Joint  Military  Education:  Actions  Needed  to  Implement  DOD  Recommendations  for 
Enhancing  Leadership  Development” 

RECOMMENDATION  1:  To  guide  the  implementation  of  actions  DOD  identified  in  its  study 
on  JPME,  GAO  recommends  that  the  Chairman  of  the  Joint  Chiefs  of  Staff  direct  the  Director  for 
Joint  Force  Development,  to  establish  well-defined  timeframes  for  conducting  follow-on  actions, 
coordinate  with  all  stakeholders,  and  identify  key  officials  responsible  for  implementing  the 
study’s  recommendations  to  help  ensure  the  usefulness,  timeliness,  and  implementation  of  any 
actions  DOD  takes  in  response  to  the  findings  and  recommendations  contained  in  its  study. 

DOD  RESPONSE:  The  DOD  concurs  with  this  recommendation.  As  the  Joint  Staff  pursues 
implementation  of  Review  of  Joint  Education  (RJE)  recommendations,  the  Joint  Staff  will 
establish  timelines,  identify  Offices  of  Primary  Responsibility,  and  coordinate  with  appropriate 
stakeholders  as  appropriate.  This  effort  is  already  in  progress,  with  the  first  phase  of  activity 
dedicated  to  updating  the  CJCS’  Professional  Military  Education  (PME)  policies  to  ensure  that 
the  policy  directives  driving  PME  are  consistent  with  the  results  and  recommendations  of  the 
RJE.  Specifically,  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  (CJCSI)  1 800.01 ,  Officer 
Professional  Military  Education  Policy  (OPMEP)  and  CJCSI  1805.01 ,  Enlisted  Professional 
Military  Education  Policy  (EPMEP),  are  already  in  the  process  of  being  revised  with  a  targeted 
completion  of  summer  2014. 

RECOMMENDATION  2:  To  guide  the  implementation  of  actions  DOD  identified  in  its  study 
on  JPME,  GAO  recommends  that  the  Chairman  of  the  Joint  Chiefs  of  Staff  direct  the  Director  for 
Joint  Force  Development,  to  assess  the  costs  of  implementing  recommendations  made,  and 
efficiencies  to  be  derived  from  the  recommendations  in  order  to  implement  DOD’s 
recommendations  in  a  cost-effective  manner. 

DOD  RESPONSE:  The  DOD  concurs  with  this  recommendation;  it  should  however  be  noted 
that  the  recommendation  expands  the  RJE  beyond  its  original  scope  and  purpose.  This  said, 
given  the  reality  of  resourcing  today,  the  Joint  Staff  will,  inherent  to  the  decision  process, 
consider  costs  and  efficiencies  prior  to  moving  forward  with  implementation  of  any  RJE 
recommendations.  In  addition,  the  RJE  final  report  recognized  current  resource  constraints  and 
stated  that,  to  the  maximum  extent  possible,  implementation  of  recommendations  should  take 
place  via  OPMEP  and  EPMEP  processes,  using  established  procedures  as  opposed  to  creating 
separate  efforts  for  implementation.  A  number  of  the  recommendations  will  be  implemented  via 
policy  changes,  which  are  cost  neutral.  As  the  primary  focus  of  the  RJE  was  on  curriculum, 
potential  changes  will  also  be  made  within  existing  curriculum  review  processes.  Using  standing 
processes  will  lead  to  cost  efficiencies.  The  Services  will  also  consider  costs  and  efficiencies  for 
recommendations  under  their  purview. 
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Appendix  V:  GAO  Contact  and  Staff 
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GAO  Contact 


Brenda  S.  Farrell  (202)  512-3604  orfarrellb@gao.gov. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (http://www.gao.gov).  Each  weekday 
afternoon,  GAO  posts  on  its  website  newly  released  reports,  testimony, 
and  correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted 
products,  go  to  http://www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates  Listen  to  our  Podcasts. 
Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 
7125,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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